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CLAIMS 

1. An image processing method of generating an image 
of a virtual space formed from a virtual object 
including at least one part, characterized by 
5 comprising: 

a first acquisition step of acquiring a position 
and orientation of a viewpoint of an observer; 

■ 

a secpnd acquisition step of acquiring a position 
and orientation of a pointing device which is worn by 
10 the observer on a hand to execute various kinds of 
operations; 

a layout step of laying out a list image to 
display a list of pieces of information about the parts 
near the position acquired in the second acquisition 
15 step; 

* 

a virtual space image generation step of 
generating the image of the virtual space after laying 
out the list image, which is seen in accordance with 
the position and orientation of the viewpoint; and 
20 an output step of outputting the image generated 

in the virtual space image generation step to a 
predetermined display device. 

2 • The method according to claim 1 , characterized in 
25 that in the layout step, the list image is laid out 
while making a plane of the list image, on which the 
pieces of information about the parts are described. 
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visible from the position and orientation of the 
viewpoint . 

3. The method according to claim 1, characterized in 
5 that in the layout step, the list image is laid out at 

a position that internally divides a line segment 
connecting the position acquired in the first 
acquisition step and the position acquired in the 
second acquisition step to t : (1-t), * 

10 

4. The method according to claim 1, characterized by 
further comprising a layout control step of controlling 
processing in the layout step in accordance with an 
instruction whether to display the list image to switch 

15 whether to contain the list image in the virtual space 
image . 

5. The method according to claim 1, characterized by 
further comprising a conversion step of converting the 

20 list image into a semitransparent image in accordance 
with an instruction to change the list image to the 
semitransparent image . 

6 . The method according to claim 1 , characterized by 

■ 

25 further comprising 

a determination step of determining on the basis 
of the position and orientation of the viewpoint and a 
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position of the virtual object whether the virtual 
object is present in a direction of line of sight of 
the viewpoint, and 

a transparency control step of , when it is 
5 determined in the determination step that the virtual 
object is present, making a transparency of the list 
image higher than that when it is determined in the 
determination step that the virtual object is not 
present . 

10 

7 . The method according to claim 1 , characterized by 
further comprising 

a distance calculation step of calculating a 
distance between the position of the pointing device 
15 and the position of the virtual object, and 

a list image generation step of generating the 
list image to display the list of pieces of information 
about the parts up to a layer level corresponding to 
the distance calculated in the distance calculation 

■ 

20 step in a hierarchical structure of the parts included 
in the virtual object, 

8 . The method according to claim 1 , characterized in 
that the list image is an image to display a list of 

25 pieces of information about, of the parts included in 

the virtual object, a part at a position closest to the 
position of the pointing device acquired in the second 
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acquisition step. 

9. The method according to claim 1 # characterized in 
that in the virtual space image generation step, when 

5 the list image overlaps a hand region in the physical 
space image acquired in the physical space image 
acquisition step, the image of the virtual space is 
generated on the basis of priority data to designate 
which of the hand region and the list image should be 
10 rendered in front. 

* 

10. The method according to claim 9, characterized by 
further comprising a designation step of designating 
which of the hand region and the list image should be 

15 rendered in front, 

wherein in the designation step, designated 
contents are set to the priority data. 

11. An image processing method of generating a 
20 virtual image corresponding to a position and 

orientation of an observer and compositing the virtual 
image with a physical image corresponding to the 
position and orientation of the observer, including 
acquiring the position and orientation of the 

25 observer, 

generating the virtual image in accordance with 
the position and orientation of the observer, and 
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compositing the physical image corresponding to 
the position and orientation of the observer with the 
virtual image, characterized by comprising: 

acquiring a position of pointing means operated 

5 by the observer; 

identifying a part pointed by the pointing means; 
generating a part information virtual image to 

i 

display information about the identified part; and 
determining a layout position of the part 
10 information virtual image in accordance with the 
position of the pointing means. 

12. The method according to claim 11, characterized 
in that the layout position is determined in accordance 

15 with the position of the pointing means and the 
position and orientation of the observer. 

13. The method according to claim 11, characterized 
in that the information about the part comprises a list 

20 of a plurality of parts associated with the identified 
part . 

14. An image processing apparatus of generating an 
image of a virtual space formed from a virtual object 

25 including at least one part, characterized by 

r 

comprising: 

first acquisition unit adapted to acquire a 
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position and orientation of a viewpoint of an observer; 

second acquisition unit adapted to acquire a 
position and orientation of a pointing device which is 
worn by the observer on a hand to execute various kinds 
5 of operations; 

layout unit adapted to lay out a list image to 
display a list of pieces of information about the parts 

■ 

near the position acquired by said second acquisition 
unit; 

10 virtual space image generation unit adapted to 

generate the image of the virtual space after laying 
out the list image, which is seen in accordance with 
the position and orientation of the viewpoint; and 

output unit adapted to output the image generated 

15 by said virtual space image generation unit to a 
predetermined display device. 

15. An image processing apparatus of generating a 

virtual image corresponding to a position and 
20 orientation of an observer and compositing the virtual 

image with a physical image corresponding to the 

position and orientation of the observer, comprising: 
acquiring unit adapted to acquire the position 

and orientation of the observer, 
25 generating unit adapted to generate the virtual 

image in accordance with the position and orientation 

of the observer, and 
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compositing unit adapted to composite the 
physical image corresponding to the position and 
orientation of the observer with the virtual image, 
characterized by comprising: 
5 acquiring unit adapted to acquire a position of 

pointing means operated by the observer; 

■ • 

identifying unit adapted to identify a part 
pointed by the pointing means; 

generating unit adapted to generate a part 
10 information virtual image to display information about 
the identified part; and 

determining unit adapted to determining a layout 
position of the part information virtual image in 
accordance with the position of the pointing means. 

15 

16. A program characterized by causing a computer to 
execute an image processing method of claim 1. 

17. A program characterized by causing a computer to 
20 execute an image processing method of claim 11. 



